Meta-scheduler with meta-contexts

ABSTRACT

A process in a computer system creates and uses a meta-scheduler with meta-contexts that execute on meta-virtual processors. The meta-scheduler includes a set of schedulers with scheduler-contexts that execute on virtual processors. The meta-scheduler schedules the scheduler-contexts on the meta-contexts and schedules the meta-contexts on the meta-virtual processors which execute on execution contexts associated with hardware threads.

BACKGROUND

Processes executed in a computer system may include schedulers thatschedule tasks of processes for execution in the computer system. Ascheduler may access operating system (OS) execution contexts (e.g.,threads, fibers, or child processes) in order to execute tasks onprocessing resources allocated to the scheduler.

A process may create any number of schedulers where each scheduleroperates with any number of execution contexts. As the number ofschedulers increases, the overall number of execution contexts used bythe schedulers may become difficult to manage. In addition, an excessnumber of overall execution contexts may have an undesirable impact onthe execution of the process by the computer system.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

A process in a computer system creates and uses a meta-scheduler withmeta-contexts that execute on meta-virtual processors. Themeta-scheduler includes a set of schedulers with scheduler-contexts thatexecute on virtual processors. The meta-scheduler schedules thescheduler-contexts on the meta-contexts and schedules the meta-contextson the meta-virtual processors which execute on execution contextsassociated with hardware threads.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of embodiments and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments andtogether with the description serve to explain principles ofembodiments. Other embodiments and many of the intended advantages ofembodiments will be readily appreciated as they become better understoodby reference to the following detailed description. The elements of thedrawings are not necessarily to scale relative to each other. Likereference numerals designate corresponding similar parts.

FIGS. 1A-1B are block diagrams illustrating embodiments of ameta-scheduler with meta-virtual processors and meta-contexts in aruntime environment.

FIGS. 2A-2B are flow charts illustrating embodiments of methods forexecuting meta-contexts in a meta-scheduler.

FIGS. 3A-3D are block diagrams illustrating embodiments of meta-contextsin a meta-scheduler.

FIG. 4 is a block diagram illustrating an embodiment of a schedulinggroup for use in a scheduler of a meta-scheduler.

FIG. 5 is a block diagram illustrating an embodiment of a computersystem configured to implement a runtime environment including ameta-scheduler with meta-virtual processors and meta-contexts.

DETAILED DESCRIPTION

In the following Detailed Description, reference is made to theaccompanying drawings, which form a part hereof, and in which is shown,by way of illustration, specific embodiments in which the invention maybe practiced. In this regard, directional terminology, such as “top,”“bottom,” “front,” “back,” “leading,” “trailing,” etc., is used withreference to the orientation of the Figure(s) being described. Becausecomponents of embodiments can be positioned in a number of differentorientations, the directional terminology is used for purposes ofillustration and is in no way limiting. It is to be understood thatother embodiments may be utilized and structural or logical changes maybe made without departing from the scope of the present invention. Thefollowing detailed description, therefore, is not to be taken in alimiting sense, and the scope of the present invention is defined by theappended claims.

It is to be understood that the features of the various exemplaryembodiments described herein may be combined with each other, unlessspecifically noted otherwise.

FIGS. 1A-1B are block diagrams illustrating embodiments of ameta-scheduler 18 with meta-virtual processors 20(1)-20(P) andcorresponding meta-contexts 21(1)-21(P) (viz., thread proxies) in aruntime environment 10, where P is an integer that is greater than orequal to one and denotes the Pth meta-virtual processor 20(P) andmeta-context 21(P). Each meta-context 21 provides quanta of execution ofa corresponding meta-virtual processor 20 to scheduler-contexts 34 oncorresponding virtual processors 32 across a set of schedulers22(1)-22(N) of the meta-scheduler 18, where N is an integer that isgreater than or equal to one and denotes the Nth scheduler 22(N).

In FIG. 1A, runtime environment 10 represents a runtime mode ofoperation in a computer system, such as a computer system 100 shown inFIG. 5 and described in additional detail below, where the computersystem is executing instructions. The computer system generates runtimeenvironment 10 from a runtime platform such as a runtime platform 122shown in FIG. 5 and described in additional detail below.

Runtime environment 10 includes an least one invoked process 12, aresource management layer 14, and a set of hardware threads 16(1)-16(M),where M is an integer that is greater than or equal to one and denotesthe Mth hardware thread 16(M). Runtime environment 10 allows tasks fromprocess 12 to be executed, along with tasks from any other processesthat co-exist with process 12 (not shown), using an operating system(OS) such as an OS 120 shown in FIG. 5 and described in additionaldetail below, resource management layer 14, and hardware threads16(1)-16(M). Runtime environment 10 operates in conjunction with the OSand/or resource management layer 14 to allow process 12 to obtainprocessor and other resources of the computer system (e.g., hardwarethreads 16(1)-16(M)).

Runtime environment 10 includes a meta-scheduler function that generatesmeta-scheduler 18 with meta-virtual processors 20 and meta-contexts 21and a scheduler function that generates schedulers 22 for inclusion inmeta-scheduler 18. In one embodiment, the meta-scheduler and schedulerfunctions are implemented as application programming interfaces (APIs).In other embodiments, one or more of the functions may be implementedusing other suitable programming constructs. When invoked, themeta-scheduler function creates meta-scheduler 18 to manage one or moreschedulers 22 in meta-scheduler 18. The meta-scheduler function alsocreates a set of meta-virtual processors 20(1)-20(P), where eachmeta-virtual processor 20 manages a corresponding set of virtualprocessors 32 across the schedulers 22 in meta-scheduler 18. Themeta-scheduler function further creates a set of meta-contexts21(1)-21(P), where each meta-context 21 executes a corresponding set ofscheduler-contexts across the schedulers 22 in meta-scheduler 18 on acorresponding meta-virtual processor 20. The scheduler function createsa scheduler 22 in meta-scheduler 18, where each scheduler 22 operates toschedule scheduler-contexts of process 12 for execution on virtualprocessors 32 of the scheduler 22. The scheduler-contexts execute onmeta-contexts 21 which in turn execute on execution contexts on hardwarethreads 16. Runtime environment 10 may exploit fine grained concurrencythat application or library developers express in their programs (e.g.,process 12) using accompanying tools that are aware of the facilitiesthat the meta-scheduler, meta-context, and scheduler functions provide.

Process 12 includes an allocation of processing and other resources thathosts execution contexts (e.g., threads, fibers, or child processes).Process 12 obtains access to the processing and other resources in thecomputer system (e.g., hardware threads 16(1)-16(M)) from the OS and/orresource management layer 14. Process 12 causes tasks to be executedusing the processing and other resources.

Process 12 generates work in tasks of variable length where each task isassociated with a scheduler-context in scheduler 22. Each task includesa sequence of instructions that perform a unit of work when executed bythe computer system. Each scheduler-context includes program state andmachine state information that is saved when the scheduler-contextblocks, yields, or is otherwise interrupted. Scheduler-contexts mayterminate or enter an idle or sleep state when there are no more tasksleft to execute. For each task, runtime environment 10 and/or process 12either assign the task to scheduler 22 to be scheduled for execution orotherwise cause the task to be executed without using scheduler 22.

Process 12 may be configured to operate in a computer system based onany suitable execution model, such as a stack model or an interpretermodel, and may represent any suitable type of code, such as anapplication, a library function, or an operating system service. Process12 has a program state and machine state associated with a set ofallocated resources that include a defined memory address space. Process12 executes autonomously or substantially autonomously from anyco-existing processes in runtime environment 10. Accordingly, process 12does not adversely alter the program state of co-existing processes orthe machine state of any resources allocated to co-existing processes.Similarly, co-existing processes do not adversely alter the programstate of process 12 or the machine state of any resources allocated toprocess 12.

Resource management layer 14 allocates processing resources to process12 by assigning one or more hardware threads 16 to process 12. Resourcemanagement layer 14 also includes the meta-scheduler function in oneembodiment and thus, creates and manages meta-scheduler 18, meta-virtualprocessors 20, and meta-contexts 21. Resource management layer 14 causesmeta-contexts 21 on corresponding meta-virtual processors 20 to beexecuted on underlying execution contexts obtained from the OS onhardware threads 16. Resource management layer 14 exists separately fromthe OS in the embodiments of FIGS. 1A and 1B. In other embodiments,resource management layer 14 or some or all of the functions thereof maybe included in the OS or runtime environment 10.

Hardware threads 16 reside in execution cores of a set or one or moreprocessor packages (e.g., processor packages 102 shown in FIG. 5 anddescribed in additional detail below) of the computer system. Eachhardware thread 16 is configured to execute instructions independentlyor substantially independently from the other execution cores andincludes a machine state. Hardware threads 16 may be included in asingle processor package or may be distributed across multiple processorpackages. Each processor package may include hardware threads 16 withthe same or different architectures and/or instruction sets. Forexample, hardware threads 16 may include any suitable combination ofin-order cores, superscalar cores, and general purpose graphicalprocessing unit (GPGPU) cores. Each execution core in a processorpackage may include one or more hardware threads 16.

Process 12 implicitly or explicitly causes meta-scheduler 18,meta-virtual processors 20(1)-20(P), meta-contexts 21(1)-21(P), andschedulers 22(1)-22(N) to be created via the corresponding functionsprovided by runtime environment 10 and/or resource management layer 14.Meta-scheduler 18, meta-virtual processors 20, meta-contexts 21, andschedulers 22 may be implicitly created when process 12 uses APIsavailable in the computer system or programming language features. Inresponse to the API or programming language features, runtimeenvironment 10 creates meta-scheduler 18, meta-virtual processors 20,meta-contexts 21, and schedulers 22 that inherit the policies ofmeta-scheduler 18. To explicitly create meta-scheduler 18, meta-virtualprocessors 20, meta-contexts 21, and schedulers 22, process 12 mayinvoke the meta-scheduler and scheduler functions provided by runtimeenvironment 10 and specify one or more policies for meta-scheduler 18,meta-virtual processors 20, meta-contexts 21, and schedulers 22.

Meta-scheduler 18 manages meta-virtual processors 20 and correspondingmeta-contexts 21 to share meta-virtual processors 20 and meta-contexts21 among all schedulers 22(1)-22(N) in meta-scheduler 18. Meta-scheduler18 may share meta-virtual processors 20 and meta-contexts 21 amongschedulers 22(1)-22(N) cooperatively, preemptively, or with anothersuitable type of time slicing. As part of creating meta-scheduler 18,resource management layer 14 allocates meta-virtual processors 20 andmeta-contexts 21 to meta-scheduler 18 based on supply and demand and anypolicies of meta-scheduler 18. In one embodiment, meta-scheduler 18creates each scheduler 22(1)-22(N). In other embodiments, one or more ofscheduler 22(1)-22(N) that are external to meta-scheduler 18 may invokea programming API or other suitable programming construct to attach tometa-scheduler 18.

In one embodiment, process 12 adds each scheduler 22(1)-22(N) tometa-scheduler 18 with the same set of scheduler policies. In anotherembodiment, process 12 adds each scheduler 22(1)-22(N) to meta-scheduler18 with a different set of scheduler policies. Each scheduler 22receives virtual processors 32(1)-32(P) where each virtual processor 32forms an abstraction of underlying meta-virtual processors 20 andhardware threads 16. Each scheduler 22 also receives information thatmaps virtual processors 32(1)-32(P) of a scheduler 22 to correspondingmeta-virtual processors 20(1)-20(P). As shown in FIG. 1B, virtualprocessors 32(1)(1)-32(N)(1) from respective schedulers 22(1)-22(N) mapto meta-virtual processor 20(1), virtual processors 32(1)(2)-32(N)(2)from respective schedulers 22(1)-22(N) map to meta-virtual processor20(2), and so on.

Meta-scheduler 18 allows meta-virtual processors 20 and meta-contexts 21to be shared among scheduler-contexts 34 of schedulers 22(1)-22(N)cooperatively, preemptively, or with another suitable time slicing. Eachmeta-virtual processor 20 forms an abstraction of a hardware thread 16and executes a corresponding meta-context 21. Each meta-context 21 formsan abstraction of a scheduler-context and executes the scheduler-contexton a corresponding meta-virtual processor 20. Resource management layer14 multiplexes meta-virtual processors 20 onto hardware threads 16 bymapping each meta-virtual processor 20 to a hardware thread 16. Resourcemanagement layer 14 may map more than one meta-virtual processor 20 ontoa particular hardware thread 16 but maps only one hardware thread 16 toeach meta-virtual processor 20. In other embodiments, resourcemanagement layer 14 manages processing resources in other suitable waysto cause meta-contexts 21 to be executed by hardware threads 16.

Meta-scheduler 18 schedules the scheduler-contexts on meta-contexts 21and schedules meta-contexts 21 on meta-virtual processors 20 whichexecute on execution contexts associated with hardware threads 16. Eachmeta-context 21 switches between execution of scheduler-contexts 34 onvirtual processors 32 on the corresponding meta-virtual processor 20.Each meta-context 21 causes a single scheduler-context 34 to be executedat any given time but periodically performs context switches betweenexecution of scheduler-contexts 34 to execute each of the set ofscheduler-contexts 34 on virtual processors 32 that correspond to themeta-virtual processor 20 of the meta-context 21. Each meta-context 21provides a quantum of execution upon dispatching a scheduler-context 34of a scheduler 22. The quantum of execution may be expressed in time(e.g., 50 ms), by a number of tasks to be executed, or by any othersuitable metric. The quantum of execution may be the same or differentfor each dispatched scheduler-context 34.

As shown in FIG. 1B, meta-context 21(1) switches between execution ofscheduler-contexts 34(1)(1)-34(N)(1) from respective schedulers22(1)-22(N), meta-context 21(2) switches between execution ofscheduler-contexts 34(1)(2)-34(N)(2) from respective schedulers22(1)-22(N), and so on. As shown by an arrow 44, for example,meta-context 21(1) dispatches scheduler-context 34(1)(1) for a quantumof execution on meta-virtual processor 20(1) and, once scheduler-context34(1)(1) detects that quantum has expired and yields back tometa-context 21(1), meta-context 21(1) dispatches scheduler-context34(2)(1) for a quantum of execution on meta-virtual processor 20(1).Meta-context 21(1) continues the process of dispatching a next one ofthe set of scheduler-contexts 34(1)(1)-34(N)(1) each time a current oneof the set of scheduler-context 34(1)(1)-34(N)(1) yields back tometa-context 21(1).

Referring back to FIG. 1A, scheduler 22 executes scheduler-contexts 34on virtual processors 32 which are, in turn, executed by meta-contexts21 on meta-virtual processors 20. The set of scheduler-contexts in eachscheduler 22 includes a set of scheduler-contexts 34 with respective,associated tasks 36 that are being executed by virtual processors 32and, at any point during the execution of process 12, a set of zero ormore runnable scheduler-contexts 38 and a set of zero or more blocked(i.e., wait-dependent) scheduler-contexts 40. Each scheduler-context 34,38, and 40 includes state information that indicates whether ascheduler-context 34, 38, or 40 is executing, runnable (e.g., inresponse to becoming unblocked or added to a scheduler 22), or blocked.Scheduler-contexts 34 that are executing have been attached to a virtualprocessor 32 and are currently executing. Scheduler-contexts 38 that arerunnable include an associated task 39 and are ready to be executed byan available virtual processor 32. Scheduler-contexts 40 that areblocked also include an associated task 41 and are waiting for data, amessage, or an event that is being generated by anotherscheduler-context 34 or will be generated by another scheduler-context38 or 40.

Each scheduler-context 34 executing on a virtual processor 32 maygenerate, in the course of its execution, additional tasks 42, which areorganized in any suitable way (e.g., added to work queues (not shown inFIGS. 1A and 1B)). Work may be created by using either applicationprogramming interfaces (APIs) provided by runtime environment 10 orprogramming language features and corresponding tools in one embodiment.When processing resources are available in a scheduler 22, tasks areassigned to scheduler-contexts 34 or 38 that execute them to completionon virtual processors 32 before picking up new tasks. Ascheduler-context 34 executing on a virtual processor 32 may alsounblock other scheduler-contexts 40 by generating data, a message, or anevent that will be used by other scheduler-contexts 40.

Each task in each scheduler 22 may be realized (e.g., realized tasks 36and 39), which indicates that a scheduler-context 34 or 38 has been orwill be attached to the task and the task is ready to execute. Realizedtasks typically include light-weight tasks and agents and may beassociated with a scheduler-context 34 or 38 just before executing or inadvance of execution. A task that is not realized is termed unrealized.Unrealized tasks (e.g., tasks 42) may be created as child tasksgenerated by the execution of parent tasks and may be generated byparallel constructs (e.g., parallel or parallel for). Each scheduler 22may be organized into a synchronized collection (e.g., a stack and/or aqueue) for logically independent tasks with scheduler-contexts (i.e.,realized tasks) along with a list of workstealing queues for dependenttasks (i.e., unrealized tasks) as illustrated in the embodiment of FIG.4 described below.

Prior to executing tasks, scheduler 22 creates scheduler-contexts 34 and38 for execution on meta-contexts 21. Available virtual processors 32locate and execute scheduler-contexts 34 to begin executing tasks.Virtual processors 32 become available again in response to ascheduler-context 34 completing, blocking, or otherwise beinginterrupted (e.g., explicit yielding or forced preemption). When virtualprocessors 32 become available, the available virtual processor 32 mayswitch to a runnable scheduler-context 38 to execute an associated task39. The available virtual processor 32 may also execute a next task 39or 42 as a continuation on a current execution context 34 if theprevious task 36 executed by the current execution context 34 completed.

Each scheduler 22 searches for a runnable execution context 38, arealized task 39, or an unrealized task 42 to attach to the availablevirtual processor 32 for execution in any suitable way. For example, ascheduler 22 may search for a runnable execution context 38 to executebefore searching for a realized task 39 or an unrealized task 42 toexecute. Each scheduler 22 continues attaching scheduler-contexts 38 toavailable virtual processors 32 for execution until all tasks andscheduler-contexts 38 of each scheduler 22 have been executed. In otherembodiments, runnable execution contexts 38 and realized tasks 39 may bemerged into single concept from the perspective of schedulers 22.

The operation of meta-scheduler 18, meta-virtual processors 20,meta-contexts 21, and schedulers 22 in executing process 12 will now bedescribed with reference to FIGS. 2A-2B and 3A-3B. FIGS. 2A-2B are flowcharts illustrating embodiments of methods for executing meta-contexts21 in meta-scheduler 18, and FIGS. 3A-3D are block diagrams illustratingembodiments of meta-contexts 21(1) and 21(2) in meta-scheduler 18 duringthe execution of process 12.

In FIG. 2A, each meta-context 21 identifies a corresponding set ofscheduler-contexts 34 across a set of schedulers 22(1)-22(N) asindicated in a block 52. Each meta-context 21 in meta-scheduler 18maintains a dispatch aggregate 72 (e.g., dispatch aggregate 72(1) shownin FIG. 3A). Each dispatch aggregate 72 is a data structure thatidentifies the scheduler-contexts 34 attached to the virtual processors32 corresponding to the meta-virtual processor 20 of the meta-context21. As shown in the example of FIG. 3A, dispatch aggregate 72(1) ofmeta-context 21(1) identifies scheduler-contexts 34(1)(1)-34(N)(1)attached to virtual processors 32(1)(1)-32(N)(1) that correspond tometa-virtual processor 20(1).

Scheduler 22 adds virtual processors 32 to one or more meta-virtualprocessors 20 when a scheduler 22 is added to meta-scheduler 18 andremoves virtual processors 32 from one or more meta-virtual processors20 when a scheduler 22 is removed from meta-scheduler 18. In addition,schedulers 22 may also remove virtual processors 32 from one or moremeta-virtual processors 20 when virtual processors 32 enter an idle orsleep state and add virtual processors 32 to one or more meta-virtualprocessors 20 when virtual processors 32 return from an idle or sleepstate.

For each meta-virtual processor 20 with an added virtual processor 32, acorresponding meta-context 21 modifies a corresponding dispatchaggregate 72 to identify a scheduler-context 34 attached to the virtualprocessor 32 added to the meta-virtual processor 20. For eachmeta-virtual processor 20 with a removed virtual processor 32, acorresponding meta-context 21 modifies a corresponding dispatchaggregate 72 to remove the identification of the scheduler-context 34attached to the virtual processor 32 removed from the meta-virtualprocessor 20. When a virtual processor 32 enters an idle or sleep statein a scheduler 22, the scheduler 22 notifies the correspondingmeta-virtual processor 20 and meta-context 21. Because the virtualprocessor 32 is removed from the corresponding dispatch aggregate 72,meta-context 21 does not provide the virtual processor 32 with a quantumof execution until notified that the virtual processor 32 is awake andhas tasks to execute.

Meta-contexts 21 use dispatch aggregates 72 to identifyscheduler-contexts 34 to dispatch for execution. In one embodiment, eachmeta-context 21 switches between scheduler-contexts 34 of schedulers 22in a corresponding dispatch aggregate 72 in a round robin order. Inother embodiments, each meta-context 21 switches betweenscheduler-contexts 34 of schedulers 22 in a corresponding dispatchaggregate 72 in other suitable orders.

Each meta-context 21 dispatches a scheduler-context 34 of next scheduler22 as determined from a corresponding dispatch aggregate 72 as indicatedin a block 54. In the example of FIG. 3B, meta-context 21(1) maydispatch a scheduler-context 34(1)(1) of next scheduler 22(1).

Each meta-context 21 provides a quantum of execution upon dispatching ascheduler-context 34 of a scheduler 22. Each scheduler 22 that receivesa quantum of execution causes a scheduler-context 34 to be executed onthe corresponding virtual processor 32 via the correspondingmeta-context 21 and corresponding meta-virtual processor 20 for theallotted quantum.

Each scheduler 22 detects when a quantum has expired and prevents thecorresponding scheduler-context 34 from scheduling another task forexecution. Each scheduler 22 also notifies the correspondingmeta-context 21 when a quantum has expired to return execution tometa-context 21 (e.g., by the scheduler-context 34 function exiting andreturning control to meta-context 21). In one embodiment, each scheduler22 may determine if a quantum has expired each time that a taskcompletes, blocks, or is otherwise interrupted on a correspondingscheduler-context 34. If so, then scheduler 22 returns control ofexecution to meta-context 21. If not, then scheduler 22 allowsscheduler-context 34 to schedule another task for execution. In otherembodiments, each scheduler 22 and/or scheduler-context 34 may determineif a quantum has expired in other suitable ways.

During each quantum allotted to a scheduler 22 (i.e., during executionof a scheduler-context 34 by the scheduler 22), the meta-context 21detects when a scheduler 22 switches to a next scheduler-context 34 onthe corresponding virtual processor 32 as indicated in a block 56 andwhen the scheduler yields as indicated in a block 59. Scheduler 22notifies the meta-context 21 when a switch to a next scheduler-context34 is desired and when scheduler 22 yields (i.e., the quantum hasexpired).

If scheduler 22 does not switch to a next scheduler-context 34 on thecorresponding virtual processor 32, then scheduler 22 executes arealized task 39, if found, on the current scheduler-context 34 asindicated in a block 57 or an unrealized task 42 on the currentscheduler-context 34 if a realized task 39 is not found and theunrealized task 42 is found as indicated in a block 58. If neither arealized task 39 or an unrealized task 42 is found, then scheduler 22may pause before again searching for another task or scheduler-context38 to execute as indicated in blocks 56, 57, and 58 or yield asindicated in block 59.

When a scheduler 22 yields without switching to a next scheduler-context34, the meta-context 21 identifies a scheduler-context 34 of a nextscheduler 22 using the dispatch aggregate 72 and dispatches theidentified scheduler-context 34 as indicated in a block 54. In theexample of FIG. 3A, meta-context 21 may identify a scheduler-context34(2)(1) of a next scheduler 22(2) using the dispatch aggregate 72(1)and dispatch scheduler-context 34(2)(1) with a quantum to scheduler22(2).

During execution of a scheduler-context 34 by the scheduler 22,scheduler 22 may desire to switch to a next scheduler-context 34 or 38on the virtual processor 32. The scheduler-context 34 on virtualprocessor 32 may block, complete, or otherwise be interrupted, forexample, to prompt scheduler 22 to desire to make the context switch. Ifa scheduler 22 notifies meta-context 21 that a switch to a nextscheduler-context 34 or 38 (i.e., a scheduler-context 34 or 38 thatdiffers from the scheduler-context 34 dispatched by meta-context 21) isdesired, then meta-context 21 blocks as indicated in a block 60.

Any time that a meta-context 21 blocks, the corresponding meta-virtualprocessor 20 switches to another meta-context 21 as illustrated in FIG.2B. In response to detecting that a meta-context 21 blocks, runtimeenvironment 10 determines whether the next scheduler-context 34 or 38 inthe scheduler 22 performing the context switch is already bound to ameta-context 21 as indicated in a block 62.

If the next scheduler-context 34 or 38 is not bound to a meta-context21, then runtime environment 10 creates a new meta-context 21 for thenext scheduler-context 34 as indicated in a block 64. Otherwise, themeta-context 21 which is already bound to the next scheduler-context 34will be used. In both cases, runtime environment 10 binds the previousscheduler-context 34 (i.e., the scheduler-context 34 being switched outby the scheduler 22) to the blocked meta-context 21 as indicated in ablock 64. Runtime environment 10 updates the dispatch aggregate 72 ofthe next meta-context 21 as indicated in a block 68. To do so, runtimeenvironment 10 moves all of the dispatch aggregate 72 of the blockedmeta-context 21 except the previous scheduler-context 34 into thedispatch aggregate 72 of the next meta-context 21. Runtime environment10 then adds the next scheduler-context 34 or 38 and scheduler 22 intothe dispatch aggregate 72 of the next meta-context 21. Runtimeenvironment 10 associates the next meta-context 21 with thecorresponding meta-virtual processor 20 as indicated in a block 70.

FIGS. 3B-3C illustrate an example of the blocking of meta-context 21(1)and the switch to a next meta-context 21(P+1) on meta-virtual processor20(1). In FIG. 3B, scheduler 22(1) notifies meta-context 21(1) of acontext switch to scheduler-context 34(1)(2). Because scheduler-context34(1)(2) was not already bound to a meta-context 21, runtime environment10 creates a new meta-context 21(P+1) for scheduler-context 34(1)(2). InFIG. 3C, runtime environment 10 binds scheduler-context 34(1)(1) to theblocked meta-context 21(1). Runtime environment 10 updates dispatchaggregate 72(P+1) by moving all of the dispatch aggregate 72(1) exceptscheduler-context 34(1)(1) into dispatch aggregate 72(P+1) and addingscheduler-context 34(1)(2) and scheduler 22(1) into dispatch aggregate72(P+1). Runtime environment 10 associates meta-context 21(P+1) withmeta-virtual processor 20(1) and blocks meta-context 21(1).

At some later point, scheduler 22(1) may decide to switch back toscheduler-context 32(1)(1) on virtual processor 32(1). FIG. 3Dillustrates an example of the resumption of meta-context 21(1) and theblocking of meta-context 21(P+1) on meta-virtual processor 20(1). InFIG. 3D, scheduler 22(1) notifies meta-context 21(P+1) of a contextswitch to scheduler-context 34(1)(1). Runtime environment 10 bindsscheduler-context 34(1)(2) to the blocked meta-context 21(P+1) andblocks meta-context 21(P+1). Because scheduler-context 34(1)(1) isalready bound to meta-context 21(1), runtime environment 10 updatesdispatch aggregate 72(1) by moving all of the dispatch aggregate 72(P+1)except scheduler-context 34(1)(2) into dispatch aggregate 72(P+1) tojoin scheduler-context 34(1)(1) and scheduler 22(1). Runtime environment10 re-associates meta-context 21(1) with meta-virtual processor 20(1).

In the above embodiments, each meta-context 21 executes to processinformation received from schedulers 22 between dispatches ofscheduler-contexts 34. Accordingly, each meta-context 21 maintains itsown state to prevent races between schedulers 22.

Although each scheduler 22 is shown as including P virtual processors32, one or more of schedulers 22 may include fewer than or greater thanP virtual processors 32 in other embodiments. As a result, eachmeta-virtual processor 20 may have different numbers of correspondingvirtual processors 32 at various times. In addition, each meta-virtualprocessor 20 may have different numbers of corresponding virtualprocessors 32 at various times in response to virtual processors 32 inschedulers 22 entering into and resuming from idle and sleep states.

As described above, each dispatch aggregate 72 includes only thosescheduler-contexts 34 presently attached to a virtual processor 32 thatcorresponds to a meta-virtual processor 20. Accordingly, a dispatchaggregate 72 may, at various times, identify a single scheduler-context34. When this occurs, a meta-context 21 may provide an extended orinfinite quantum of execution to the corresponding scheduler 22 to allowthe scheduler 22 to execute the scheduler-context 34 for an extendedperiod without cooperatively yielding to the meta-context 21.

The above embodiments allow a desired number of meta-contexts 21 andcorresponding execution contexts (e.g., OS threads) to service a largenumber of schedulers 22. By managing the number of meta-contexts 21,runtime environment 10 may prevent an excessive number of executioncontexts from being created in process 12 and prevent the number ofcontexts from adversely impacting the execution of process 12.

In one embodiment, process 12 (shown in FIG. 1A) organizes tasks intoone or more schedule groups 90 (shown in FIG. 4) and presents eachschedule group 90 to one of schedulers 22(1)-22(N) as shown in FIG. 4.In other embodiments, process 12 organizes tasks into collections foreach virtual processor 32 of each scheduler 22 or in other suitableways.

FIG. 4 is a block diagram illustrating an embodiment of a schedule group90 for use in a corresponding scheduler 22. Schedule group 90 includes arunnables collection 92, a realized task collection 93, a workcollection 94, and a set of zero or more workstealing queues 96.Runnables collection 92 contains a list of unblocked scheduler-contexts38. A scheduler 22 adds a scheduler-context 38 to runnables collection92 when a scheduler-context 40 becomes unblocked. Realized taskcollection 93 contains a list of realized tasks 39 (e.g., unstartedagents) that may or may not have associated scheduler-contexts 38. Thescheduler 22 adds a realized task to realized task collection 93 when anew runnable task is presented to scheduler 22 by process 12. Workcollection 94 contains a list of workstealing queues 96 as indicated byan arrow 98 and tracks the scheduler-contexts 34 that are executingtasks from the workstealing queues 96. Each workstealing queue 96includes one or more unrealized tasks 42 with no assignedscheduler-context.

Using the embodiment of FIG. 4, the scheduler 22 may first search forunblocked scheduler-contexts 38 in the runnables collection 92 of eachschedule group 90 in the scheduler 22. The scheduler 22 may then searchfor realized tasks in the realized task collection 93 of all schedulegroups 90 in the scheduler 22 before searching for unrealized tasks inthe workstealing queues 96 of the schedule groups 90.

In one embodiment, a virtual processor 32 that becomes available mayattempt to locate a runnable scheduler-context 38 in the runnablescollection 92 in the schedule group 90 from which the available virtualprocessor 32 most recently obtained a runnable scheduler-context 38(i.e., the current schedule group 90). The available virtual processor32 may then attempt to locate a runnable scheduler-context 38 in therunnables collections 92 in the remaining schedule groups 90 of thescheduler 22 in a round-robin or other suitable order. If no runnablescheduler-context 38 is found, then the available virtual processor 32may then attempt to locate an unrealized task 42 in the workstealingqueues 96 of the current schedule group 90 before searching theworkstealing queues 96 in the remaining schedule groups 90 of thescheduler 22 in a round-robin or other suitable order.

In other embodiments, schedule groups 90 contain other suitable numbers,types, and/or configurations of task collections.

FIG. 5 is a block diagram illustrating an embodiment of computer system100 which is configured to implement runtime environment 10 includingmeta-scheduler 18 with meta-virtual processors 20 and meta-contexts 21.

Computer system 100 includes one or more processor packages 102, amemory system 104, zero or more input/output devices 106, zero or moredisplay devices 108, zero or more peripheral devices 110, and zero ormore network devices 112. Processor packages 102, memory system 104,input/output devices 106, display devices 108, peripheral devices 110,and network devices 112 communicate using a set of interconnections 114that includes any suitable type, number, and configuration ofcontrollers, buses, interfaces, and/or other wired or wirelessconnections.

Computer system 100 represents any suitable processing device configuredfor a general purpose or a specific purpose. Examples of computer system100 include a server, a personal computer, a laptop computer, a tabletcomputer, a personal digital assistant (PDA), a mobile telephone, and anaudio/video device. The components of computer system 100 (i.e.,processor packages 102, memory system 104, input/output devices 106,display devices 108, peripheral devices 110, network devices 112, andinterconnections 114) may be contained in a common housing (not shown)or in any suitable number of separate housings (not shown).

Processor packages 102 include hardware threads 16(1)-16(M). Eachprocessor package 102 may include hardware threads 16 with the same ordifferent architectures and/or instruction sets. For example, hardwarethreads 16 may include any combination of in-order execution cores,superscalar execution cores, and GPGPU execution cores. Each hardwarethread 16 in processor packages 102 is configured to access and executeinstructions stored in memory system 104. The instructions may include abasic input output system (BIOS) or firmware (not shown), OS 120, aruntime platform 122, applications 124, and resource management layer 14(also shown in FIG. 1A). Each hardware thread 16 may execute theinstructions in conjunction with or in response to information receivedfrom input/output devices 106, display devices 108, peripheral devices110, and/or network devices 112.

Computer system 100 boots and executes OS 120. OS 120 includesinstructions executable by hardware threads 16 to manage the componentsof computer system 100 and provide a set of functions that allowapplications 124 to access and use the components. In one embodiment, OS120 is the Windows operating system. In other embodiments, OS 120 isanother operating system suitable for use with computer system 100.

Resource management layer 14 includes instructions that are executablein conjunction with OS 120 to allocate resources of computer system 100including hardware threads 16 as described above with reference to FIG.1A. Resource management layer 14 may be included in computer system 100as a library of functions available to one or more applications 124 oras an integrated part of OS 120.

Runtime platform 122 includes instructions that are executable inconjunction with OS 120 and resource management layer 14 to generateruntime environment 10 and provide runtime functions to applications124. These runtime functions include a meta-scheduler function and ascheduler function as described in additional detail above withreference to FIGS. 1A and 1B. The runtime functions may be included incomputer system 100 as part of an application 124, as a library offunctions available to one or more applications 124, or as an integratedpart of OS 120 and/or resource management layer 14.

Each application 124 includes instructions that are executable inconjunction with OS 120, resource management layer 14, and/or runtimeplatform 122 to cause desired operations to be performed by computersystem 100. Each application 124 represents one or more processes, suchas process 12 as described above, that may execute with meta-scheduler18 as provided by runtime platform 122.

Memory system 104 includes any suitable type, number, and configurationof volatile or non-volatile storage devices configured to storeinstructions and data. Memory system 104 may include any suitable cachehierarchy, be configured as a shared or distributed memory system, andmay embody a locality scheme such as a non-uniform memory access (NUMA)scheme. In addition, memory system 104 may be configured as a singleinstruction stream multiple different memory store (SIMD) system, amultiple instruction stream multiple different memory store (MIMD)system, or a computer cluster coupled through a messaging protocol suchas concurrent read, concurrent write (CRCW), concurrent read, exclusivewrite (CREW), or parallel random access machine (PRAM).

The storage devices of memory system 104 represent computer readablestorage media that store computer-executable instructions including OS120, resource management layer 14, runtime platform 122, andapplications 124. The instructions are executable by computer system toperform the functions and methods of OS 120, resource management layer14, runtime platform 122, and applications 124 described herein.Examples of storage devices in memory system 104 include hard diskdrives, random access memory (RAM), read only memory (ROM), flash memorydrives and cards, and magnetic and optical disks.

Memory system 104 stores instructions and data received from processorpackages 102, input/output devices 106, display devices 108, peripheraldevices 110, and network devices 112. Memory system 104 provides storedinstructions and data to processor packages 102, input/output devices106, display devices 108, peripheral devices 110, and network devices112.

Input/output devices 106 include any suitable type, number, andconfiguration of input/output devices configured to input instructionsor data from a user to computer system 100 and output instructions ordata from computer system 100 to the user. Examples of input/outputdevices 106 include a keyboard, a mouse, a touchpad, a touchscreen,buttons, dials, knobs, and switches.

Display devices 108 include any suitable type, number, and configurationof display devices configured to output textual and/or graphicalinformation to a user of computer system 100. Examples of displaydevices 108 include a monitor, a display screen, and a projector.

Peripheral devices 110 include any suitable type, number, andconfiguration of peripheral devices configured to operate with one ormore other components in computer system 100 to perform general orspecific processing functions.

Network devices 112 include any suitable type, number, and configurationof network devices configured to allow computer system 100 tocommunicate across one or more networks (not shown). Network devices 112may operate according to any suitable networking protocol and/orconfiguration to allow information to be transmitted by computer system100 to a network or received by computer system 100 from a network.

Although specific embodiments have been illustrated and describedherein, it will be appreciated by those of ordinary skill in the artthat a variety of alternate and/or equivalent implementations may besubstituted for the specific embodiments shown and described withoutdeparting from the scope of the present invention. This application isintended to cover any adaptations or variations of the specificembodiments discussed herein. Therefore, it is intended that thisinvention be limited only by the claims and the equivalents thereof.

1. A method comprising: providing a first quantum of execution on afirst meta-virtual processor to a first scheduler-context of a firstscheduler in a meta-scheduler created by a process executing on acomputer system; and providing a second quantum of execution on thefirst meta-virtual processor to a second scheduler-context of a secondscheduler in the meta-scheduler subsequent to the first quantum ofexecution on the first meta-virtual processor.
 2. The method of claim 1wherein the first scheduler-context is attached to a first virtualprocessor of the first scheduler, and wherein the secondscheduler-context is attached to a second virtual processor of thesecond scheduler.
 3. The method of claim 1 further comprising:identifying the first scheduler-context and the first scheduler in adispatch aggregate prior to providing the first quantum of execution;and identifying the second scheduler-context and the second scheduler inthe dispatch aggregate prior to providing the second quantum ofexecution.
 4. The method of claim 1 further comprising: providing athird quantum of execution on a second meta-virtual processor to a thirdscheduler-context of the first scheduler during the first quantum ofexecution; and providing a fourth quantum of execution on the secondmeta-virtual processor to a fourth scheduler-context of the secondscheduler subsequent to the first quantum of execution on the secondmeta-virtual processor.
 5. The method of claim 4 wherein the firstscheduler-context is attached to a first virtual processor of the firstscheduler, wherein the third scheduler-context is attached to a secondvirtual processor of the first scheduler, wherein the secondscheduler-context is attached to a third virtual processor of the secondscheduler, and wherein fourth second scheduler-context is attached to afourth virtual processor of the second scheduler.
 6. The method of claim1 further comprising: detecting that a third scheduler is added to themeta-scheduler; and adding a third scheduler-context and the thirdscheduler to a dispatch aggregate corresponding to the firstmeta-virtual processor.
 7. The method of claim 6 further comprising:providing a third quantum of execution on the first meta-virtualprocessor to the third scheduler-context of the third scheduler in themeta-scheduler subsequent to the second quantum of execution on thefirst meta-virtual processor.
 8. The method of claim 1 furthercomprising: subsequent to providing the first quantum of execution,detecting that a virtual processor that executes the firstscheduler-context has entered an idle state; and subsequent to providingthe second quantum of execution, providing a third quantum of executionon the first meta-virtual processor to the second scheduler-context ofthe second scheduler in the meta-scheduler prior to providing a fourthquantum of execution on the first meta-virtual processor to the firstscheduler subsequent to the first scheduler-context returning from theidle state.
 9. A method performed by a process executing on a computersystem, the method comprising: creating a meta-scheduler with first andsecond meta-virtual processors and corresponding first and secondvirtual processor contexts; creating a first scheduler with first andsecond virtual processors in the meta-scheduler and a second schedulerwith third and fourth virtual processors in the meta-scheduler; andassociating the first and the third virtual processors with the firstmeta-virtual processor and the second and the fourth virtual processorswith the second meta-virtual processor.
 10. The method of claim 9further comprising: executing a first scheduler-context on the firstvirtual processor in response to receiving a first quantum of executionfrom the first meta-context; and subsequent to the first quantum ofexecution, executing a second scheduler-context on the third virtualprocessor in response to receiving a second quantum of execution fromthe first meta-context.
 11. The method of claim 10 further comprising:yielding to the first meta-context in response to the first quantum ofexecution expiring; and receiving the second quantum of executionsubsequent to yielding to the first meta-context.
 12. The method ofclaim 10 further comprising: during the first quantum of execution,executing a third scheduler-context on the second virtual processor inresponse to receiving a third quantum of execution from the secondmeta-context; and subsequent the third quantum of execution, executing afourth scheduler-context on the fourth virtual processor in response toreceiving a fourth quantum of execution from the second meta-context.13. The method of claim 12 further comprising: yielding to the secondmeta-context in response to the third quantum of execution expiring; andreceiving the fourth quantum of execution subsequent to yielding to thesecond meta-context.
 14. The method of claim 9 further comprising:executing a first scheduler-context on the first virtual processor inresponse to receiving a first quantum of execution from the firstmeta-context; notifying the first meta-context of a context switch fromthe first scheduler-context to a second scheduler-context on the firstvirtual processor; and executing the second scheduler-context on thefirst virtual processor in response to receiving a second quantum ofexecution from a third meta-context.
 15. The method of claim 14 furthercomprising: during the first quantum of execution, executing a thirdscheduler-context on the second virtual processor in response toreceiving a third quantum of execution from the second meta-context. 16.A computer readable storage medium storing computer-executableinstructions that, when executed by a computer system, perform a methodcomprising: executing a first meta-context on a first meta-virtualprocessor in a meta-scheduler that switches between execution of atleast a first scheduler-context on a first virtual processor of a firstscheduler and a second scheduler-context on a second virtual processorof a second scheduler; and executing a second meta-context on a secondmeta-virtual processor in the meta-scheduler that switches betweenexecution of at least a third scheduler-context on a third virtualprocessor of the first scheduler and a fourth scheduler-context on afourth virtual processor of the second scheduler.
 17. The computerreadable storage medium of claim 16, the method further comprising:executing a third meta-context on the first meta-virtual processor inthe meta-scheduler that switches between execution of at least a fifthscheduler-context on the first virtual processor of the first schedulerand the second scheduler-context on the second virtual processor of thesecond scheduler in response to the first scheduler switching to thefifth scheduler-context on the first virtual processor.
 18. The computerreadable storage medium of claim 17, the method further comprising:binding the first scheduler-context on the first virtual processor tothe first meta-context in response to the first scheduler switching tothe fifth scheduler-context on the first virtual processor; and blockingthe first meta-context.
 19. The computer readable storage medium ofclaim 17, the method further comprising: creating the third meta-contextin response to the fifth scheduler-context not being bound to the thirdmeta-context.
 20. The computer readable storage medium of claim 17, themethod further comprising: executing a fourth meta-context on the secondmeta-virtual processor in the meta-scheduler that switches betweenexecution of at least the third scheduler-context on the third virtualprocessor of the first scheduler and a sixth scheduler-context on thefourth virtual processor of the second scheduler in response to thesecond scheduler switching to the sixth scheduler-context on the fourthvirtual processor.